From Bad Models to Good Policies: an Intertwined Story about Energy and Reinforcement Learning

Batch mode reinforcement learning is a subclass of reinforcement learning for which the decision making problem has to be addressed without model, using trajectories only (no model, nor simulator nor additional interactions with the actual system). In this setting, we propose a discussion about a minmax approach to generalization for deterministic problems with continuous state space. This approach aims at computing robust policies considering the fact that the sample of trajectories may be arbitrarily bad. This discussion will be intertwined with the description of a fascinating batch mode reinforcement learning-type problem with trajectories of societies as input, and for which crucial good decisions have to be taken: the energy transition

Automatic phase identification of smart meter measurement data

peer reviewedThis paper highlights the importance of the knowledge of the phase identification for the different measurement points inside a low-voltage distribution network. Besides considering existing solutions, we propose a novel method for identifying the phases of the measurement devices, based exclusively on voltage measurement correlation. It relies on graph theory and the notion of maximum spanning tree. It has been tested on a real Belgian LV network, first with simulated unbalanced voltage for which it managed to correctly identify the phases of all measurement points, second, on preliminary data from a real measurement campaign for which it shows encouraging results.PREMASO

Variable selection for dynamic treatment regimes: a reinforcement learning approach

peer reviewe

Planification Optimiste dans les Processus Décisionnels de Markov avec Croyance

Cet article décrit l'algorithme BOP (de l'anglais ``Bayesian Optimistic Planning''), un nouvel algorithme d'apprentissage par renforcement Bayésien indirect (c'est à dire fondé sur un modèle). BOP étend l'approche de l'algorithme OP-MDP (de l'anglais ``Optimistic Planning for Markov Decision Processes'', voir [Busoniu2011,Busoniu2012]) au cas où les probabilités de transitions du MDP sous-jacent sont initialement inconnues, et doivent être apprises au travers d'interactions avec l'environnement. Les connaissances sur le MDP sous-jacent sont représentées par une distribution de probabilités sur l'ensemble de tous les modèles de transitions à l'aide de distributions de Dirichlet. L'algorithme BOP planifie dans l'espace augmenté état-croyance obtenu par concaténation du vecteur d'état avec la distribution postérieure sur les modèles de transitions. On montre que BOP atteint l'optimalité Bayésienne lorsque le paramètre de budget tend vers l'infini. Quelques expériences préliminaires montrent des résultats encourageants.Peer reviewe

Foreseeing New Control Challenges in Electricity Prosumer Communities

peer reviewedThis paper is dedicated to electricity prosumer communities, which are groups of people producing, sharing and consuming electricity locally. This paper focuses on building a rigorous mathematical framework in order to formalise sequen- tial decision making problems that may soon be encountered within electricity prosumer communities. After introducing our formalism, we propose a set of optimisation problems reflecting several types of theoretically optimal behaviours for energy exchanges between prosumers. We then discuss the advantages and disadvantages of centralised and decentralised schemes and provide illustrations of decision making strategies, allowing a prosumer community to generate more distributed electricity (compared to commonly applied strategies) by mitigating over- voltages over a low-voltage feeder. We finally investigate how to design distributed control schemes that may contribute reaching (at least partially) the objectives of the community, by resort in to machine learning techniques to extract, from centralised solution(s), decision making patterns to be applied locally. First empirical results show that, even after a post-processing phase so that it satisfies physical constraints, the learning approach still performs better than predetermined strategies targeting safety first, then cost minimisation

Towards CO2 valorization in a multi remote renewable energy hub framework

In this paper, we propose a multi-RREH (Remote Renewable Energy Hub) based optimization framework. This framework allows a valorization of CO2 using carbon capture technologies. This valorization is grounded on the idea that CO2 gathered from the atmosphere or post combustion can be combined with hydrogen to produce synthetic methane. The hydrogen is obtained from water electrolysis using renewable energy (RE). Such renewable energy is generated in RREHs, which are locations where RE is cheap and abundant (e.g., solar PV in the Sahara Desert, or wind in Greenland). We instantiate our framework on a case study focusing on Belgium and 2 RREHs, and we conduct a techno-economic analysis. This analysis highlights the key role played by the cost of the two main carbon capture technologies: Post Combustion Carbon Capture (PCCC) and Direct Air Capture (DAC). In addition, we use our framework to derive a carbon price threshold above which carbon capture technologies may start playing a pivotal role in the decarbonation process of our industries. For example, this price threshold may give relevant information for calibrating the EU Emission Trading System so as to trigger the emergence of the multi-RREH